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REAL PARTY IN INTEREST 

The real party in interest is Creatier Interactive, LLC by virtue of an Assignment from the 
U.S. Bankruptcy Court to Creatier Interactive LLC, recorded on Reel/Frame 015036/0672. 

RELATED APPEALS AND INTERFERENCES 
There are no other appeals or interferences known to the Appellants or the Appellants' 
representative, which are believed to directly affect or be directly affected by or have a bearing 
on the Board's decision in the pending appeal. 
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STATUS OF CLAIMS 

Claims 31-38 are pending. The final rejection of claims 31-38 form the basis for this 
appeal. Claims 31-36 have been rejected under 35 U.S.C. §102(e) as being anticipated by 
Rangan et al., U.S. patent no. 6,198,833. Claims 37 and 38 have been rejected un 35 U.S.C. 
§ 103(a) as being unpatentable over Rangan et al., U.S. patent no. 6,198,833 in view of Issadore- 
Barreca et al., U.S. Patent No. 6,205,231 . 

STATUS OF AMENDMENTS 

All amendments have been entered. The claims as currently amended are attached as an 
Appendix. 

SUMMARY OF THE INVENTION 

The present invention relates to an automatic authoring system for interactive television. 
In such interactive television systems, it is desirable to "tag" items of interest that may appear in 
the video frames which may be of interest to a viewing audience. Such interactive television 
systems allow the user to click on the tagged item during the broadcast for additional information 
on the item of interest. In order to provide such capability, items of interest have to be tagged in 
all video frames where the items of interest may appear. In a NTSC broadcast, video frames are 
broadcasted at a rate of about 30 frames per second. Thus, a one-hour broadcast would involve 
108,000 video frames. Tagging the 108,000 video frames for the various items of interest is an 
arduous task, to say the least. In order to solve this problem, automatic authoring systems have 
been developed. These systems allow a user to tag items of interest in a single video frame and 
the system automatically locates and tags these items in succeeding video frames. 

The present invention relates to a relatively simple method and apparatus for tagging 
items of interest in video frames by associating a color 120 of an object with an event. In 
particular, the system determines the locations within each video frame where a selected item 
may appear. The system determines the color value of the selected item by its digital color 
mathematic properties. For example, one embodiment of the invention utilizes the RGB 
standard which separates the red 220, green 230, and blue 240 properties as the color as 
numerical values 250. Another embodiment uses the HSV characteristics of the color which 
define the hue 320, saturation 330 and value 340 and components of color. 
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Once an item of interest is determined in an initial frame, the system then automatically 
locates the selected item based on the color mathematical characteristics in the remaining frames 
in the broadcast. These items are then tagged and linked to an event, such as a URL. One 
embodiment of the invention uses an edge detection scheme to define the boundaries for the 
colors of the selected object. An important aspect of the present invention is that the invention 
utilizes a relatively simple process for otherwise completing a rather arduous task. 

ISSUES ON APPEAL 

I. Should the Board reverse the rejection of claims 31-36 as being anticipated by 
Rangan et al., U.S. patent no. 6,198,833 (hereinafter referred to as "Rangan.") 

II. Should the Board reverse the rejection of claims 37 and 38 as being unpatentable 
over Rangan et al., U.S. patent no. 6,198,833 in view of Issadore-Barreca et al., U.S. Patent No. 
6,205,231 (hereinafter referred to as "Issadore-Barreca.") 

GROUPING OF CLAIMS 

It is respectfully submitted that the claims 3 1 -36 stand or fall together. Also, claims 37 
and 38 stand or fall together. 

ARGUMENTS 

I. REJECTION OF CLAIMS 31-36 UNDER 35 U.S.C. §102(E) AS BEING 
UNPATENTABLE OVER RANGAN ET AL. 

Claims 31 and 32 have been rejected under 35 U.S.C. § 102(e) as being anticipated by 

Rangan, et al., U.S. Patent No. 6,198,833. In order for there to be anticipation, each and every 
one of the elements of the claims must be found in a single reference. It is respectfully submitted 
that Claims 31 and 32, as well as Claims 33-38, include elements clearly not suggested or 
disclosed in the Rangan, et al. patent. For example, Claim 31 recites, in combination, a method 
which includes a step for determining the location where an action by a pointing device has 
occurred. This claim also recites the step of determining a color value with the selected location 
and automatically associating an event with the color value in succeeding video frames. Claim 
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32 is similar. Claim 35 recites a system in which the locations of an object selected by a pointing 
device are determined. This claim also recites determining a color range for the selected object 
and associating an event with the color range in succeeding video frames. Claim 34 is similar. 
Claims 35 and 36 are similar but relate to a color pattern. Finally, Claims 37 and 38 are similar 
but relate to an edge of a selected object. The Rangan patent does not disclose such elements. 

Rangan, on the other hand, tracks a rectangular wire frame, having a fixed size around the 
location where the pointing device was exercised by comparing the color of individual pixels in 
the fixed wire frame from one frame to the next. Rangan does not define an object based on its 
color, nor does it dynamically track the object based on color as in the claims at issue. Rangan 
tracks objects based on the movement of individual pixels from frame to frame within a wire 
frame. This movement tracking is based on the initial wire frame location which is tracked in 
each frame and does not automatically associate a color with an object as in the claims at issue. 
Tracking movement of the wire frame in Rangan demands testing of individual coordinates 
against their counterpart from the previous frame. This testing requires significant data to be 
computed at each frame. Further, it is respectfully submitted that Rangan's color signature are 
not representative of a continuous color pattern. In particular, Rangan's color signature records 
the color value of every pixel in a fixed area rather than detecting a color pattern based on a 
region of interest. As such, it should be clear that claims 31-36 disclose elements clearly not 
disclosed or suggested by Rangan. Thus, the Board is respectfully requested to reverse this 
rejection. 
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II. REJECTION OF CLAIMS 37-38 UNDER 35 U.S.C. $1 03(A) AS BEING 
UNPATENTABLE OVER RANGAN ET AL. IN VIEW OF 
ISSADORE-BARRECA ET AL. 

Claims 37 and 38 recite, in combination, a method and system for automatically 
identifying a bounded object based on color characteristics in a series of video frames and 
associating an event with said object. It is respectfully submitted that claims 37 and 38 are not 
suggested by Rangan or Issadore-Barreca. As discussed above, Rangan does not track objects in 
the same manner as the system in the current claims. In particular, Rangan tracks the movement 
of an object of fixed shape by comparing the color of individual pixels in a fixed matrix from one 
frame to the next. Nowhere does Rangan disclose or suggest tracking an object based solely on 
its color, nor does it dynamically change the shape and size of the object based on color. 

The Issadore-Barreca, et al. patent was cited for disclosing an edge detection scheme. 
However, the Examiner has failed to show that an edge detection scheme would be compatible 
with the fired wire frame tracking method taught by Rangan. Indeed, both systems would appear 
to be incompatible with each other. 

Also, it is respectfully submitted that the Examiner has failed to establish a prima facie 
case of obviousness as set forth in MPEP § 2142 and § 2143. In order to establish a prima facie 
case of obviousness, three criteria must be met as set forth in MPEP § 2143. 

"First, there must be some suggestion or motivation, either in the references 
themselves or in the knowledge generally available to one of ordinary skill in the 
art, to modify the reference or to combine reference teachings. Second, there 
must be a reasonable expectation of success. Finally, the prior art reference (or 
references when combined) must teach or suggest all the claim limitations. 

The teaching or suggestion to make the claimed combination reasonable 
expectation of success must both be found in the prior art, not in the Applicant's 
disclosure." 

It is respectfully submitted that the examiner is combining two references in support of the 
rejections without a showing that the motivation for the combination was suggested by the 
reference itself. Moreover, none of the references, either singly or in combination, teach all of 
the elements of the claims. For example, all of the claims recite automatically associating an 
object with its color in a plurality of video frames. None of the references cited disclose such an 
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automatic authoring system. Moreover, there is no reasonable expectation that the combined 
references would succeed in an interactive TV application. For all of the above reasons, the 
Board is respectfully requested to reverse the Examiner's rejection of claims 37 and 38. 
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CONCLUSION 



The Board is respectfully requested to reverse the rejection of the claims by the 



Katten Muchin Rosenman LLP 
525 West Monroe Street 
Chicago, Illinois 60661 
Tel: (312) 902-5200 
Fax: (312) 902-1061 



Examiner. 



Respectfully submitted, 



KATTEN MUCHIN ROSENMAN LLP 
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t&Aue^ APPENDIX A 

CLAIMS ON APPEAL 

31. A method for automatically identifying an object in a plurality of video frames 
and associating the object with an event comprising: 

determining a location in one of said video frames where an action by a pointing device 
has occurred defining a selected location; 

determining a color value of said selected location; and 

automatically associating an event with said color value of said selected location in said 
one video frame and automatically associating events with said color value in succeeding video 
frames. 

32. A system for automatically identifying an object in a series of video frames and 
associating an event with said object, the system comprising: 

a system for determining a location in one video frame where an action by a pointing 
device has occurred defining a selected location; 

a system for determining a color value associated with said selected location in said one 
video frame; and 

a system for automatically associating an event with said color value in succeeding video 

frames. 

33. A method for automatically identifying an object in a plurality of video frames 
and associating the object with an event comprising: 

determining the coordinates of an object in a video frame that is selected by a pointing 
device defining a selected object; 

determining a color range for said selected object; and 
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automatically associating an event with said color range of said selected location in said 
one video frame and automatically associating events with said color value in succeeding video 
frames. 

34. A system for automatically identifying an object in a series of video frames and 
associating an event with said object, the system comprising: 

a system for determining the coordinates of an object in one video frame that has been 
selected by a pointing device defining a selected object; 

a system for determining a color range for said selected object in said one video frame; 

and 

a system for automatically associating an event with said color range in said one video 
frame and succeeding video frames. 

35. A method for automatically identifying an object in a plurality of video frames 
and associating the object with an event comprising: 

determining the coordinates of an object selected in one of said video frames by a 
pointing device defining a selected object; 

determining a color pattern for said selected object; and 

automatically associating an event with said color pattern of said selected location in said 
one video frame and automatically associating events with said color pattern in succeeding video 
frames. 

36. A system for automatically identifying an object in a series of video frames and 
associating an event with said object, the system comprising: 

a system for determining the coordinates of an object in one video frame selected a 
pointing device defining a selected object; 
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a system for determining a color pattern for said selected object in said one video frame; 

and 

a system for automatically associating an event with said color pattern in said one video 
frame and succeeding video frames. 

37. A method for automatically identifying an object in a plurality of video frames 
and associating the object with an event comprising: 

determining the coordinates along one edge of an object in one of said video frames 

selected by a pointing device defining a selected object; 

determining the color values along said edge of said selected object; and 

automatically associating an event with said color values along said edge in said one 

video frame and automatically associating events with said color values in succeeding video 

frames associated with the color values. 

38. A system for automatically identifying an object in a series of video frames and 
associating an event with said object, the system comprising: 

a system for determining the coordinates along one edge of an object in one video frame 
selected by a pointing device defining a selected object; 

a system for determining the color values along said edge of said selected object in said 
one video frame; and 

a system for automatically associating an event with said color values in said one video 
frame and succeeding video frames. 
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